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ABSTRACT 


Tliis  report  describes  some  interpretations  and  uses  of 
eigenvalues  and  eigenvectors  of  spectral  and  sample  spectral 
density  matrices  of  multiple  stationary  time  series. 

The  spectral  density  matrix  of  a  zero-mean  multiple 
stationary  time  series  is  defined.  Eigenvalues  and  eigenvectors 
of  the  spectral  density  matrix  are  discussed  and  principal  com¬ 
ponent  theory  is  presented.  Statistical  distribution  theory  and 
related  results  are  used  to  investigate  the  eigenvalues  of  a 
sample  spectral  density  matrix.  This  investigation  gives  methods 
for  obtaining  simultaneous  confidence  bounds  on  the  elements  of 
the  true  spectral  density  matrix  and  its  inverse,  and  also  methods 
or  obtaining  confidence  bounds  on  the  eigenvalues  of  the  true 
spectral  density  matrix. 
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EIGENVALUES  AND  EIGENVECTORS  OF 


SPECTRAL  DENSITY  MATRICES 


The  present  report  describes  some  interpretations  and  uses  of 
eigenvalues  and  eigenvectors  of  spectral  and  sample  spectral  density 
matrices  of  multiple  stationary  time  series. 


I  SPECTRAL  REPRESENTATION  OF  A  ZERO-MEAN 
MULTIPLE  STATIONARY  TIME  SERIES 


A  zero- mean  multiple  stationary  time  series  X(t)  has  the  spectral 
representation 


X(t)  = 


'xl(tf 

~  dZjM  “ 

x2(t) 

f 00  iwt 

dZ2(w) 

• 

=  /  * 

p 

_  xp(t,_ 

J  -00 

• 

• 

dZ  (b>) 

P 

(1) 


where  the  p  x  p  Hermitian  nonnegative  definite  matrix  EM  is  the 
spectral  density  matrix  at  frequency  u>  of  X(t) . 


2.  EIGENVALUES  AND  EIGENVECTORS  OF  A 
SPECTRAL  DENSITY  MATRIX 


There  exists  a  px  p  unitary  matrix  PM »  i.  e. 


?(w)  p(u»)  =  I  =  pM  P'M 


(3) 


such  that 


XjM 

x2M 


0 


P'M  E(u)  PM  = 


0 


(4) 


x  M 

p 


where  the  X^M.  (j  =  1.  •  •  •  .  p) .  »re  the  eigenvalues  (real  and  nonnegative) 
of  EM  *nd 


X .  M  >  X ,  M  >  •  •  •  >  X  M  >  0 

l  ”  t  ~  —  p  ~ 


(5) 
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Consider 


"  dWjM* 

dZ  j  M 
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=  P'M 
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dZ  M 
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(6) 


one  has 
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w  «• 

d  W  t  M 
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=  p'M  EM  PM  du>  =  I 

I  0 


du»  =■  AM  dw 
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From  (7)  01:  observes  that  the  complex  random  variables  dW^u), 
are  uncorrelated,  and  that 


dW  («) 
P 


Var  (dW.M)  =  E  |dW.(w)J2  =  X.M  dw,  (j  -  1 . p)  (l 

Let  p.(w),  (j  =  1 . P).  denoie  the  jth  column  vector  of  the  unitary 

matrix  PM*  Since  pM  i«  unitary 


PJM  PjtM 


0  if  j  i  j’ 
1  if  j  =  j1 


(j  =  1 . P) 


(9) 


Equation  (9)  expresses  that  the  column  vec  tors  p.M  of  PM  are  ortho¬ 
normal,  i.  e.  ,  they  are  orthogonal  and  of  unit  norm.  From  (4)  one  has 


EM  PM  =  PM  Aw 


(10) 


l.  e. 


EM  PjM  =  VM  PjM 


(j  =  l.  •  •  .  P) 


(ID 


so  that  the  column  vectors  p.M  of  PM  are  the  (normalised)  eigeuvectors 
of  the  spectral  density  matrix  EM  . 
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F rom  (6), 


dZ  j  M 


dW.(u)  =  p'.M 
)  ) 


,  dZ  (u>) 

L  P  . 


(j  =  1 . P) 


(12) 


so  that  dW.(u)  is.  a  linear  combination  of  dZj(w) . dZ^w)  with  co¬ 

efficients  equal  to  the  complex  conjugate  of  the  components  of  the  column 

vector  .  (j  =  1>  •  •  •  •  P)  • 


3.  PRINCIPAL  COMPONENTS 

The  dWjM.  ....  dW  (w)  are  called  principal  components.  With  the 

eigenvalues  XjM . XpM  »n  the  order  indicated  by  (5),  dW^u,)  is 

called  the  first  principal  component,  dW2(w)  the  second  principal  component, 

etc. 

Consider  the  problem  of  determining  Cjj(w).  . Cpl^ 

satisfying 


(13) 


such  that  the  linear  combination 
variance,  i.  e. ,  such  that 
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Var  (dWjM) 


Var 


p  _  \ 

P  _ 

57  C  (w)  dZ  (w)  -  E 

l#  '■  ’  / 

c  i(w)  dZ -M 

j=l  J 

maximum 


(14) 


The  general  solution  to  that  problem  is 


(w). 


cnM. 


c  ,(w) 
pi 


u  J  (w)  Pj(w) 


(15) 


where  u^w)  denotes  an  arbitrary  complex  number  of  unit  modulus  and 
Pj(w)  is  the  eigenvector  of  2(w)  corresponding  to  the  eigenvalue  Xj(w) 
described  previously.  Thus,  the  variance  of  dW^(w)  is  a  maximum  if 
and  only  if 


dW,(w)  s  UjM  dWjM 


(16) 


Furthermore,  that  maximum  variance  is 


E  |  dW  j(u»)|  ^  =  e|  dWj(w) 


E  P'jM 


dZj(w) 


l  dZp(w) 


f 


dZ  (w) . dZ  (w) 

1  P 


(17) 


P|(w)  E(w)  Pj(u)  du  =  pJM  Xj(w)  Pj(w)  dw  =  Xj(w)  dw 
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Next,  consider  the  problem  of  determining  Cj^(u),  c22<"> . cpj'“> 

satisfying 


shH2=1 


(18) 


P  _ 


such  thst  the  linear  combination  c,j(w)  dZ.(w)  =  dW?(w)  is  uncorrelated 

jTi  J 

with  dWj(w)  and  has  maximum  variance,  i.  e. ,  such  that 


E  dW'2(w)  dW ,(«)  =  0 


(19) 


and 


Var  (dW2(w))  *  E|dW2(w)|2  =  maximum 


(20) 


The  general  solution  to  the  problem  ie 

21(«),  c22(w),  cp2(w)l  *  u2(w)  Pjj(w)  (21) 

where  u2(<i>)  denotes  an  arbitrary  complex  number  of  unit  modulus  and 
(J,(«)  is  the  eigenvector  of  E(w)  corresponding  to  the  eigenvector  K2(w) 
described  previously.  Thus,  the  variance  of  dW2(w)  is  a  maximum  if 
and  only  if 


(22) 


dW£M  =  u2M  dW2(w) 


That  maximum  variance  is 


E 


dV^fu)  =  E  dW2(u>)  =  dw 


(23) 


Similarly,  the  Cjj(w),  c23(w),  ....  c^^u)  aatiafying 


(24) 


~  P  _ 

such  that  dWj(w)  CjjM  dZi(<*>)  *•  uncorrelated  with  dW  (u>)  and  dW  (w) 

j*l  ’  '  *  2 

and  has  maximum  variance  is 


[cijM. 


cj3M. 


cp3<“l 


1 


UjM  P'3(u>) 


(25) 


the  dWj(w)  possessing  maximum  variance  is 


dW3(w)  =  u3(w)  dW3(w) 


(26) 
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and  that  maximum  variance  is 


E 


dW3<«) 


K3(u)  du 


(27) 


Proceeding  inductively  in  the  manner  illustrated  above  it  l*  eeen  that 


dWrM(“' 


ViM 


dWrtl<“' 


(28) 


it  the  solution  to  the  following  problem: 

P 


Find  the  dWr+1(«)-«  pottetting  maximum  variance 

subject  to  the  constraint  that  |  r+i(w)|  2=1  »nd  d^r+i<w)  l*  nn~ 

correlated  with  dWjM.  dW2<u) . dWfM.  Furthermore,  that 


maximum  variance  is 


E|d*t*l,“l|2  '  E|dwr*iM|2  ■  Vr+1,“)  dW 


(29) 


From  the  foregoing  discussion  it  is  seen  that  the  transformation  (6) 

of  the  [dZjto.  ....  dZp(w)]  to  the  [dW^w) . dWp(w)] .  U.,  the 

transformation  to  principal  components,  is  a  transformation  to  uncorre¬ 
lated  random  variables  possessing  the  special  variance  properties  de¬ 
scribed  above.  In  some  studies  the  principal  components  with  large 


variance  may  be  of  apecial  interest.  When  the  principal  components  with 
large  variance  "account  for  moat  of  the  variability,  "  i.  e.  ,  when  the  total 
variance  of  the  other  principal  components  is  comparatively  small,  re¬ 
stricting  attention  (in  exploratory  investigations)  to  the  principal  com¬ 
ponents  with  large  variance  may  constitute  an  effective  way  of  reducing 
the  "dimensionality"  of  a  problem. 


4.  STATISTICAL  ESTIMATION  OF  EIGENVALUES  AND  EIGENVECTORS 
OF  A  SPECTRAL  DENSITY  MATRIX 


Let  E(w)  denote  a  sample  Hermitian  nonnegative  defii  'te  spectral 
density  matrix  constituting  an  estimator  of  the  true  spectral  density 

/\  /s 

matrix  EM.  The  eigenvalues  X^M,  (j  =  1 . p),  of  EM  in  descend¬ 

ing  order  are,  respectively,  estimators  for  the  eigenvalues  ^(u), 

(j  =  1,  ...,  p),  of  E(u) .  Similarly,  the  corresponding  (normalised) 

A 

eigenve  'ors  (Mw),  (j  =  1,  .  . . ,  p),  of  E(u)  are,  respectively,  estimators 
for  the  normalised  eigenvectors  p.M,  (j  =  1 . p),  of  EM  .  Under 

1  A 

suitable  hypotheses  (i.  e. ,  hypotheses  that  make  EM  a  maximum  likeli- 

A  A 

hood  estimator  for  EM),  the  X^M,  P^M,  (j  =  1,  ....  p),  are,  respectively, 
maximum  likelihood  estimators  for  X.M,  p.M,  (j  =  1 . p) . 
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5.  STATISTICAL  DISTRIBUTION  THEORY  AND  RELATED  RESULTS 
PERTAINING  TO  THE  RANDOM  EIGENVALUES  OF  A 
SAMPLE  SPECTRAL  DENSITY  MATRIX 


To  simplify  notation  the  dependence  on  the  frequency  u>  will  not  be 

✓V  A 

indicated  here.  For  example,  £(<•>)  will  simply  be  denoted  by  E,  the 
frequency  dependence  being  understood.  That  is,  it  is  understood  that 
£  is  a  spectral  density  estimator  pertaining  to  a  small  frequency  band 
centered  at  frequency  u. 

The  distributional  theory  and  related  results  described  here  are 
predicated  on 


A 

E  = 


(30) 


where  A  is  complex  Wishart  distributed  with  n  degrees  of  freedom. 

The  complex  Wisher*  distribution  with  parameters  n,  p,  E  will  be  denoted 

by  W  (E;n,  p). 

c 


Thm.  1.  If  A  has  the  distribution  W  (I;  n,  p)  with  n  >p,  then  tl.'  random 

A  A  A  C 

eigenvalues  >  .  .  •  >  Kp  >  0  of  A  have  the  joint  probability  density 

function 


p(Kj»  K2,  . 


Kp)  =  C(p,  n) 


P 

72 

li=i 


*n- 


n-p 


Z  Kj 

j=l  J 


“I 

j<k » 
j.  k=l 


A  A  U 

Ki  •  Kk 


(31) 
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where  the  "constant"  C(p,  n)  in  given  by 


C(p.  n) 


P 

n 


r  (n  -  p  +  j)  r  (j> 


-l 


(32) 


The  probability  density  function  (31)  is  defined  over  the  domain 
K,  >  K,  >  .  .  .  >  K  >0. 

1  L  "  — *  p  — 

Since  the  eigenvalues  K  of  £  are  related  to  the  eigenvalues  K. 

1  ^  j 

of  A  by  =  -K.t  (j  =  1,  . . .  ,  p),  one  may  regard  (31)  as  giving  the 
probability  density  function  of  the  (j  =  1 . p) . 


A  A 


-hm'  2‘  If  K1‘  K2‘  *•••  Kp  are  random  variables  distributed  with  the 
probability  density  function  (31),  then  for  any  constants  a,  b  such  that 
0  <  a  <  b 


Prob  £a  <Kp . Kz,  Kj  <  bj  = 


C(p,  n) 


Y0  Y1 


Y1  Y2  * 


V>  vp 


'p-i 


2p-2 


(33) 


V4 


i+j-2 


/>  o 

n-p+i 


-p+i+j-2  -v  . 

e  dv 


(i.  j  =  1.  2, 


.  P) 


where 


1 


Equation  (33)  gives  a  closed  form  expression  for  the  probability 
that  all  the  eigenvalues  of  a  random  Hcrnntian  matrix  A  distributed 
W  (I;n,  p)  with  n  >  p  will  be  between  prescribed  limits  a  and  b. 

C  " 

Equivalently,  one  may  regard  (33)  as  giving  the  probability  that  all  the 
random  eigenvalues  of  a  sample  spectral  density  matrix  £  will  be  be¬ 
tween  prescribed  limits  for  the  cise  when  the  true  spectral  density  matrix 
2  =  1-  Confidence  band  results  pertaining  to  general  spectral  density 
matrices  are  derivable  from  Thm.  2,  so  that  the  condition  that  2  =  I  is 
not  as  restrictive  as  might  appear  at  first  glance. 

Let  A*  denote  a  random  ilermitian  matrix  distributed  W  (I;  n,  p) 

where  n  >  p.  Let  ch  .  (A*)  denote  the  minimum  eigenvalue  of  A*  and 

min  ° 

ch  (A*)  denote  the  maximum  eigenvalue  of  A*.  Let  2  denote  a  gen- 
max  ° 

eral  nonsingular  (p  x  p)  spectral  dei  sity  matrix.  From  Thm.  2  for  a 
chosen  0  <  e  <  1  one  may  obtain  constants  I*  and  u*  such  that 


1  -  e  =  Prob 


[l*  < 


ch_in<A*)»  ch  a  JA*)  <  tt*] 

min  max  —  j 


The  probability  statement  (35)  is  equivalent  to 


1  -  t  =  Prob 


Pi*  <u*, 

a'a 


for  aH  nonzero  complex  (p  x  1)  vectors  a 


Now,  there  exists  a  nonsingular  (p  x  p)  matrix  M  such  that 


M'  M  =  2 


•  *  , 


Let 


a  =  Mb 


(38) 


and 


A  =  M'  A*M 


(39) 


The  Hermitian  positive  definite  A  „  di.triboted  w  „ 

Upon  substituting  ,38,  in  ,36,  and  „.i„g  ,37,  ' 


1  -  «  =  Prob 


[a*  ^  b' Ab 

1  Iz - <  U *  .  for  all 

b'Lb  — 


nonzero  complex  (pxl)  vectora  b 

(40) 


Using  (30)  one  may  atate  (40)  in  the  form 


1  -  o  =  Prob 


1 1*  K  b'Lb  u* 

L"  “b'Lb  "  n 


for  all  nonzero  complex  (pxl)  vectora  b 

(41) 

From  (41,  on.  obtain.  ,h.  simultaneous  m..,,- - a-. 


■J 


i 

l  *  =  Probl  ~  b'  £b  <  b'  Lb  <  —■  b'  Lb  ,  •imultaneoualy 

L  ” 1  f°r  all  complex  (p 


x  1)  vectors  b 
(42) 


•J 


I 


] 
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The  vectors  b  may  be  freely  ciiosen  in  (42)  with  all  bounds  on  the  result* 
ing  linear  combinations  of  the  elements  of  £  holding  simultaneously  with 
probability  1  -  e.  From  simultaneous  confidence  bounds  on  suitably 
chosen  linear  combinations  of  the  elements  of  £  one  may  obtain,  for 
e>  ample,  simultaneous  confidence  bounds  on  al^  the  spectra,  co-sp«ctra, 
and  quadrature-spectra  of  £  .  One  may  also  view  (42)  as  giving  simultane¬ 
ous  confidence  bands  for  values  of  an  Hermitian  quadratic  form  where  £ 
is  the  matrix  of  the  quadratic  form.  Equation  (42)  a-v,o  yields 


1  -  s  =  Prob 


(£)  <  X  ,  . 
P 


(43) 


where  c^m£n(^)  denotes  the  minimum  eigenvalue  of  the  sample  spectral 

A  A 

density  matrix  £  and  ch^^(£)  denotes  the  maximum  eigenvalue.  Equa¬ 
tion  (43)  is  a  confidence  bound  for  all  the  eigenvalues  of  the  true  spectral 
density  matrix  £  . 

From  (35)  one  also  obtains 


1  -  c  =  Prob  ~  7'  £"!  c  <7'  £_1  c  <  —  7'  £*  c  ,  ■imult»‘'*;!OU«»ly  for  all 
| n  ~  —  n  complex  (p  x  1)  vectors 

(44) 

The  simultaneous  confidence  band  statement  (44)  yields  confidence  bound 
results  pertaining  to  £  1  directly  analogous  to  those  for  £  described  above 
Viewing  (44)  as  giving  simultaneous  confidence  bands  for  values  of  an 
Hermitian  quadratic  form  where  £_1  is  the  matrix  of  the  quadratic  form 
may  be  especially  important.  Such  quadratic  forms  occur  in  quadratic 
signal  detection  methods. 
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-  £ 


■  Vj 


In  conclusion  it  is  noted  that  both  (42)  and  (44)  are  derived  from  (35) 


and  hold  simultaneously,  i.  e. 


'  n  —  ^  .  n  r.d,  simultaneously  for  all 

$  b'  Eb  <b  Eb  <  — b  Eb  .  complex  (p  x  , )  vec,or.  b 


1  -  «  =  Prob 


*  *- 


I*  —  ■*_  1  —  -l  u*  —  £-1  simultaneDunly  for  all 

—  c'E  c<c'L  c  <  n  c  Z  c,  compiex  (p  x  i)  vectors  c 
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